Interest Pixel Mining

Visual media data such as an image is the raw data representation for many important applications, such as image retrieval (Mikolajczyk & Schmid 2001), video classification (Lin & Hauptmann, 2002), facial expression recognition (Wang & Ahuja 2003), face recognition (Zhao, Chellappa, Phillips & Rosenfeld 2003), etc. Reducing the dimensionality of raw visual media data is highly desirable since high dimensionality may severely degrade the effectiveness and the efficiency of retrieval algorithms. To obtain low-dimensional representation of visual media data, we can start by selecting good low-level features, such as colors, textures, and interest pixels (Swain & Ballard 1991; Gevers & Smeulders 1998; Schmid, Mohr & Bauckhage 2000). Pixels of an image may hold different interest strengths according to a specific filtering or convolution technique. The pixels of high interest strengths are expected to be more repeatable and stable than the pixels of low interest strengths across various imaging conditions, such as rotations, lighting conditions, and scaling. Interest pixel mining aims to detect a set of pixels that have the best repeatability across imaging conditions. (An algorithm for interest pixel mining is called a detector.) Interest pixel mining can be formulated into two steps: i) interest strength assignment via a specific filtering technique; and ii) candidate selection. The second step, candidate selection, plays an important role in preventing the output of interest pixels from being jammed in a small number of image regions in order to achieve best repeatability. Based on interest pixels, various image representations can be derived. A straightforward scheme is to represent an image as a collection of local appearances—the intensities of neighboring pixels—of interest pixels (Schmid & Mohr 1997). By ignoring the spatial relationship of interest pixels, this “unstructured” representation requires no image alignment, i.e., free from establishing pixel-to-pixel correspondence among imaging objects by image transformations such as rotation, translation, and scaling. Furthermore, the unstructured representation is very robust with respect to outlier regions in a retrieval application. However, the retrieval cost under unstructured representation is extremely expensive. In the context of face recognition, feature distribution is introduced to capture both global and local information of faces (Li, Ye & Kambhamettu 2006A). A limitation of feature distribution is the assumption of image alignment. A promising trend on interest pixel based representation is to build graph or tree representation for each image and measure the similarity of two images by the edit distance of their graphs or trees (Zhang & Shasha 1989). But as we will see in the later section, this trend is strongly supported by a recently proposed interest pixel mining method (Li, Ye & Kambhamettu 2008).